(19) 




Europaisches Palentamt 
European Patent Office 
Office europeen des brevets 



(12) 



(id EP 0 762 777 A2 

EUROPEAN PATENT APPLICATION 



(43) Date of publication: 

12.03.1997 Bulletin 1997/11 

(21) Application number: 96306397.9 

(22) Date of filing: 04.09.1996 



(51) IntCI* H04N7/50 



(84) Designated Contracting States: 
DE FR GB 

(30) Priority: 04.09.1995 JP 226777/95 
12.09.1995 JP 234183/95 
29.11.1995 JP 310668/95 

(71) Applicant: SHARP KABUSHIKI KAISHA 
Osaka 545 (JP) 

(72) Inventors: 

• Uchiumi, Tadashi 
Chiba-shi, Chiba (JP) 



• Iwamoto, Akira 
Chiba-shi, Chiba (JP) 

• Shioi, Masahiro 
Chiba-shi, Chiba (JP) 

(74) Representative: Brown, Kenneth Richard et al 
R.G.C. Jenkins & Co. 
26 Caxton Street 
London SW1H0RJ (GB) 



(54) Picture reproducing apparatus 

(57) A picture reproducing apparatus transmits cod- 
ed picture data and an algorithm for reproducing (de- 
coding) the coded picture data and reconfigures a de- 
coding processor in accordance with the received algo- 
rithm. The apparatus includes a unit for receiving the 
picture reproducing algorithm described in a logical de- 
scription language, performing picture reproduction in 



accordance with the received algorithm, and reconfig- 
uring a picture decoder in accordance with the received 
algorithm, a unit for adding a test routine to the algorithm 
in order to perform an operation test of the received al- 
gorithm in a program download manner, and a picture 
encoder for selecting an optimum coding system for 
each input frame and transmitting a coding system de- 
scriptor code. 
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Description 

BACKGROUND OF THE INVENTION 

(1) Field of the Invention 

The present invention relates to a picture reproduc- 
ing apparatus capable of receiving coded picture or vid- 
eo data from a network and decoding and reproducing 
the.picture,_More .specifically, the- present invention re- 
latesJo an encoding/decoding apparatus comprising^ao 
encoder for transmitting coded data and a tool consti- 
tuting an algorithm as a means for decoding the data 
and a decoder for reconfiguring the received tool as an 
algorithm and decoding the received coded data in a 
program download manner. The present invention also 
relates to a picture encoder for constructing a software 
download system by transmitting a coding system each 
time a communication line is connected when transmit- 
ting coded picture data to another terminal through the 
communication line. 

(2) Description of the Prior Art 

Recently, a picture communication service is real- 
ized as a new communication service with the spread 
of ISDN (Integrated Services Digital Network). Exam- 
ples are video phones and video conference systems. 
Also, with the development of mobile transmission net- 
works represented by PHS and FPLMTS, demands on 
the delicacy, diversity, and portability of services provid- 
ed are rapidly increasing. 

Generally, in systems such as video phones or vid- 
eo conference systems in which picture information is 
transmitted, the amount of picture information is enor- 
mous. Therefore, when the speed and the cost of lines 
used in transmission are taken into consideration, it is 
necessary to decrease the amount of picture informa- 
tion to be transmitted by compressing (coding) the pic- 
ture information before transmitting it. 

As coding systems for compressing picture infor- 
mation, JPEG (Joint Photographic coding Experts 
Group) as a still picture coding system, H. 261 as a mov- 
ing picture coding system, and MPEG1 and MPEG2 
(Moving Picture coding Expert Group) as storage type 
moving picture coding systems are already internation- 
ally standardized. In addition, MPEG4 as a coding sys- 
tem at a very low bit rate of 64 kbps or less is being 
standardized. 

MPEG4 can flexibly cope with various applications • 
and provide an optimum encoding system for each ap- 
plication. Therefore, unlike the existing JPEG, H. 261, 
MPEG1, and MPEG2 encoding systems in which en- 
coding is performed in accordance with an algorithm, 
MPEG4 must be an encoding system in which large ^ 
numbers of tools (transformers, quantizers, inverse 
transformers, and inverse quantizers or dequantizers) 
of an encoder are provided and encoding is performed 



by properly combining these tools. 

Fig. 1 A is a view showing the format of a data se- 
quence or stream of coded data obtained by encoding 
(compressing) picture data in accordance with H. 261. 
s Each coded data such as motion vector information, a 
DCT coefficient, or a quantization step shown in Fig.1 A 
is picture information encoded (compressed) on the ba- 
sis of a fixed encoding algorithm in an encoder. A de- . 
coder includes a fixed decoding algorithm with respect 
io to this encoding algorithm and decodes each received 
coded data. 

Fig.lB is a view showing the format of a..data .se- 
quence of coded data obtained when an algorithm such 
as MPEG4 encodes (compresses) picture data by using 
a flexible encoding system. The coded data sequence 
shown in Fig.1 B consists of coded (compressed) picture 
information such as motion vector information 2, a trans- 
form coefficient 4, motion vector information 6, atrans- 
form coefficient 8, and a quantization step 10, and tool 

20 information such as a motion compensation tool 1 , an 
inverse transform tool 3, a motion compensation tool 5, 
an inverse transform tool 7, and a quantization tool 9 for 
decoding the respective corresponding picture informa- 
tion. Each tool information such as the motion* com pen - 

zs sation tool 1 can be selected from a plurality of different 
pieces of tool information, and any arbitrary combination 
of these pieces of tool information can be selected. Ac- 
cordingly, the encoder transmits the picture information 
and the tool information used in the encoding to the de- 

30 coder, and the decoder decodes the received coded pic- 
ture information by using the tool information transmit- 
ted from the encoder. 

Methods of realizing these encoding/decoding 
processes include a method using dedicated hardware 

35 and software and a method which executes appropriate 
software by using a general-purpose arithmetic unit and 
a compiler. 

The method which realizes the encoding process 
by using dedicated hardware and software will be de- 

40 scribed below, Fig.2 is a block diagram showing the con- 
figuration of an encoder which generates the coded data 
shown in Fig. 1 A in accordance with H. 261 . Referring to 
Fig.2, this encoder comprises an encoding controller 11 
tor controlling encoding, a transformer 12 tor calculating 

*s DCTs, a quantizer 1 3 for quantizing the coefficient trans- 
formed by the transformer 12, an inverse quantizer 14 
for inversely quantizing the coefficient quantized by the 
quantizer 13, an inverse transformer 15 for calculating 
inverse DCTs, a memory 16, and a loop filter 17. The 

; ° memory 16 has a motion compensation variable delay- 
ing function used in motion compensation inter-frame 
prediction. The filter 17 is a loop filter which can be 
turned on or off for each macroblock. 

In realizing an encoding algorithm for generating 

5 the coded data shown in Fig.1 A by using dedicated 
hardware and software, the functions of tools constitut- 
ing this algorithm are achieved by dedicated hardware 
and software, i.e., the encoding controller 11, the trans- 
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former 12, the quantizer 13, the inverse quantizer 14, 
the inverse transformer 15, the memory 1 6 having the 
motion compensation delaying function, and the loop fil- 
ter 17. Fig. 3 is a block diagram showing the configura- 
tion of a decoder for decoding coded data encoded in 
accordance with H. 261. This decoder shares the com- 
ponents of the encoder shown in Fig.2. The same ref- 
erence numerals as in Fig.2 denote the same parts in 
Fig.3. That is, the decoder shown in Fig.3 comprises the 
inverse quantizer 14, the inverse transformer 15, the 
memory 16 having the motion compensation variable 
delaying function, and the loop filter 1 7. 

The coded data encoded by the encoder shown in 
Fig.2 is inversely quantized by the inverse quantizer 14 
and decoded by calculating inverse DCTs by the inverse 
transformer 15. The memory 16 and the loop filter 17 
are used in decoding motion compensated predictive 
coded data. 

To process several different algorithms by using the 
system which performs encoding by using a fixed algo- 
rithm such as H. 261 , dedicated hardware and software 
must be provided for each algorithm. Fig.4 is a block 
diagram showing the structure ol an encoder which en- 
codes moving pictures in accordance with H. 261 and 
encodes still pictures in accordance with JPEG. As an 
example, when one terminal encodes moving pictures 
and still pictures in accordance with H. 261 and JPEG, 
respectively, the configuration of the encoder is as 
shown in Fig.4. That is, the encoder includes both an H. 
261 encoder 20 and a JPEG encoder 21. In Fig.4, the 
H. 261 encoder 20 and the JPEG encoder 21 receive 
moving picture data and still picture data, respectively, 
and output respective coded data as compressed data. 

To realize an algorithm for generating the coded da- 
ta shown in Fig.lB by using dedicated hardware and 
software, an encoder for realizing this algorithm is 
achieved by replacing the configuration of a circuit block 
18 of the encoder shown in Fig.2 with a configuration 
shown in Fig. 5. That is, this encoder comprises a plu- 
rality of different tools in each of the transformer 12, the 
quantizer 13, the inverse quantizer 14, and the inverse 
transformer 1 5 and performs encoding by selecting nec- 
essary tools from these tools (transformer tools A to X, 
quantizer tools A to X, inverse quantizer tools A to X, 
and inverse transformer tools A to X in Fig.5). 

A decoder for decoding the coded data shown in 
Fig.lB is accomplished by replacing a circuit block 19 
constituting the decoder shown in Fig.3 with a circuit 
block 22 shown in Fig.5. That is, this decoder comprises 
a plurality of different tools in each of the inverse quan- 
tizer 14 and the inverse transformer 15 and performs 
decoding by selecting necessary tools from these tools 
(inverse quantizer tools A to X and inverse transformer 
tools A to X in Fig.5). 

In this decoding process, the tool information shown 
in Fig.lB, i.e., the motion compensation tool 1, the in- 
verse transform tool 3, the motion compensation tool 5, 
the inverse transform tool 7, and the quantization tool 9, 



are transmitted to a controller 23. The picture informa- 
tion following these tool information, i.e., the motion vec- 
tor information 2 t the transform coefficient 4, the motion 
vector information 6, and the transform coefficient 8, are 
s transmitted to the respective corresponding tools for 
processing these picture information. The controller 23 
selects tools to be used (Irom the inverse quantizers A 
to X and the'inverse transformers A to X in Fig.5) on the 
basis of the tool information. Each picture data is decod- 
es ed by using the tools selected by the controller 23. 

The method of realizing decoding by executing ap- 
propriate software by using a generaUpurpose arithme- 
tic unit and a compiler will be described below. Decoding 
of coded data with the structure shown in Fig.1 B will be 

15 described with reference to Fig.6. Fig.6 is a block dia- 
gram showing the structure of a decoder consisting of 
a general-purpose arithmetic processor 24 and a com- 
piler 25. The tool information shown in Fig.lB, i.e., the 
motbn compensation tool 1 , the inverse transform tool 

20 3, the motion compensation toot 5, the transform tool 7, 
and the quantization tool 9, are supplied to the compiler 
25 which in turn generates a processing program for 
controlling the operation of the general-purpose arith- 
metic processor 24. The picture information following 

25 these pieces of tool information, i.e., the motion vector 
information 2, the transform coefficient 4, the motion 
vector information 6, the transform coefficient 8, and the 
quantization step 10, are supplied to the general-pur- 
pose arithmetic processor 24. In accordance with the 

30 processing program generated by the compiler 25. the 
general-purpose arithmetic processor 24 processes the 
coded picture information following the tool information, 
decodes the coded data, and generates decoded data. 
Each tool information transmitted from the encoder 

35 contains not only information indicating the type of tool 
but also a tool main body which describes the procedure 
of the tool. Consequently, even if the decoder is not 
equipped with necessary tools, the received coded data 
can be decoded. 

40 |f, however, there is a transmission error in trans- 
mitting a tool main body, the received tool main body 
itself does not normally operate in some instances when 
coded data is decoded by using this transmitted tool 
main body. Generally, the error rate rises in data trans- 
mission using mobile transmission networks. If an error 
occurs in data, such as information of a tool main body, 
which describes the procedure itself, not only the data 
processing of the system is not normally executed but 
a fatal operation failure sometimes occurs in the whole 
50 system. 

A general method by which this is prevented is to 
check whether received data has an error by adding a 
parity to transmission data. That is, a parity is added in 
units ol certain transmission blocks, and an arithmetic 
55 operation is performed for this parity and a data main 
body. On the basis of the result of the arithmetic opera- 
tion, whether the received data has an error is deter- 
mined on the receiver side, ii the receiver side deter- 
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mines that there is no error, the receiver side returns 
data indicating a reception success. If the receiver side 
deler mines that there is an error, the receiver side re- 
turns data indicating a reception failure. When receiving 
the data indicating a reception success, the transmitter s 
side transmits the next block. When receiving the data 
indicating a reception failure, the transmitter side again 
transmits the current block. This procedure is repeated 
in units of transmission blocks. 

Fig.7 is a flow chart for explaining an operation al- io 
gorithm for error determination in conventional decod- 
ers. This algorithm is used to perform error determina- 
tion when a data sequence having the structure shown 
in Fig.lB is received. 

When coded data is received, whether a tool 1 con- is 
tained in this coded data has an error is checked (step 
SI). If it is determined that the tool 1 has an error (No 
in step S1), a request to retransmit the tool 1 is output 
to the transmission side (step S2), and the flow waits for 
retransmitted data. If there is no error in the too| 1 (Yes 20 
in step S1 ), whether a tool 2 has an error is checked 
(step S3). If it is determined that the tool 2 has an error 
(No in step S3), a request to retransmit the tool 2 is out- 
put to the transmission side (step S4), and the flow waits 
for retransmitted data. If there is no error in the tool 2 
(Yes in step S3), the received coded data is decoded by 
using these tools 1 and 2 (step S5), and the flow waits 
until the next coded data is input. 

When the next coded data is received, whether a 
tool 3 contained in this coded data has an error is 
checked (step S6). If it is determined that the tool 3 has 
an error (No in step S6), a request to retransmit the tool 
3 is output to the transmission side (step S7), and the 
flow waits for retransmitted data. If there is no error in 
the tool 3 (Yes in step S6), whether a tool 4 has an error 
is checked (step S8). If it is determined that the tool 4 
has an error (No in step S8), a request to retransmit the 
tool 4 is output to the transmission side (step S9), and 
the flow waits for retransmitted data. If there is no error 
in the tool 4 (Yes in step S8) : whether a tool 5 has an 
error is checked (step S10). If it is determined that the 
tool 5 has an error (No in step S1 0), a request to retrans- 
mit the tool 5 is output to the transmission side (step 
S11 ), and the flow waits for retransmitted data. If there 
is no error in the tool 5 (Yes in step S10), the received 
coded data is decoded by using these tools 3, 4 and 5 
(step S12), and the flow waits until the next coded data 
is input. 

In the error determination algorithm described 
above, if even one tool in which an error is detected ex- so 
ists, no decoding processing is executed by using other 
normal tools. That is, no decoding is performed until no 
more error is detected in all tools. Also, if an error is de- 
tected in a certain transmission block in the tool 1 con- 
tained in the received coded data (No in step S1 ), the ss 
retransmission procedure of the tool 1 (step S2) is com- 
pleted, and error determination is again executed for the 
retransmitted tool (step S1). Unless it is determined that 
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the tool 1 is normally received, the encoder does not 
transmit the next transmission block (tool 2). 

As a consequence, if the number of retransmission 
procedures increases, the operating efficiency on the 
reception side significantly decreases. This also de- 
creases the use efficiency of the communication trans- 
mission path. Furthermore, even when a tool is normally 
transmitted, whether the tool normally operates on the 
receiver side can be checked by neither the transmitting 
terminal nor the receiving terminal. 

Conventionally, moving pictures are reproduced on 
general-purpose computers such as personal comput- 
ers as well as the encoding/decoding apparatus shown 
in Figs.2 and 3. Since a plurality of picture data can be 
reproduced on a single computer by using a plurality of 
picture reproducing software, this computer can be re- 
garded as a general-purpose picture reproducing appa- 
ratus. 

Conventional dedicated picture reproducing appa- 
ratuses including the above prior art have a high picture 
reproducibility. However, to reproduce picture data to be 
reproduced by different picture reproducing algorithms, 
it is necessary to prepare different picture reproducing 
apparatuses corresponding to these picture data. 

A general-purpose picture reproducing apparatus is 
configured such that software of a specific picture repro- 
ducing algorithm is executed on general-purpose hard- 
ware. Therefore, a plurality of picture reproducing algo- 
rithms can be executed on single hardware by preparing 
software for each of different picture reproducing algo- 
rithms. The disadvantage of these general-purpose pic- 
ture reproducing apparatuses is a lower picture repro- 
ducibility than that of dedicated picture reproducing ap- 
paratuses. 

As described earlier, the picture coding system at a 
very low bit rate is being actively, internationally stand- 
ardized as MPEG4 (Moving Picture Experts Group 4). 
Also, many new coding systems have been developed 
for picture coding techniques such as a DCT system and 
a Wavelet transform system. It is considered that the 
number of coding systems keeps increasing in the fu- 
ture. 

The purpose of this MPEG4 is to give a coding sys- 
tem expandability and versatility independent of ma- 
chines. As a consequence, a software download type 
system is constructed. Accordingly, an MSDL (MPEG4 
Syniax Description Language) is constructed which 
generally describes a picture data encoding/decoding 
method, a method of transmission of the encoding/de- 
coding method, security in transmission, and a trans- 
mission error countenmeasure. 

Fig.8 shows the components of the MSDL. A coding 
system descriptor code which is the description of a pic- 
ture data encoding/decoding method is constituted by a 
plurality of profiles. One profile consists of a plurality of 
algorithms, and one algorithm consists of a plurality of 
tools. Thus the coding system descriptor code has a tree 
structure, and these tools, configuration methods of the 
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tools, and intertool interfaces are described by the 
MSDL. Also, in the existing coding system descriptor 
codes, a new coding system descriptor code can be 
constituted by the MSDL by exchanging or adding tools. 

The MPEG 4 defines Configuration Phase, Learn- 
ing Phase, and Transmission Phase during communi- 
cation, and the contents of. processing are determined 
for eachpftase. In Configuration Phase, to start call set- 
ting or establish a communication path, whether an en- 
coder and a decoder can perform appropriate commu- 
nication is checked. Forthis purpose, parameters de- 
scribing, e g., capability, conditions, and environment 
are exchanged. Also, after these parameters are ex- 
changed, an environment for communication is deter- 
mined or whether the control is to proceed on to next 
Learning Phase is checked. In Learning Phase, to allow 
the decoder to correctly analyze coded data transmitted 
from the encoder, transmission of an analysis method 
itself or construction of the analysis method is designat- 
ed. In Transmission Phase, coded data is transmitted 
by the format specified in these two phases. 

Fig. 9 shows a conventional picture encoding appa- 
ratus of the MPEG4 system. In Fig.9, reference numeral 
31 denotes a communication state controller; 32, a pic- 
ture encoder: and 33, a coding system descriptor code 
transmitter. The communication state controller 31 con- 
trols the state of communication and, in accordance with 
the communication state, selects transmission of picture 
coded data or transmission of a coding system descrip- 
tor code. The picture encoder 32 encodes an input pic- 
ture and forms picture coded data. The coding system 
descriptor code transmitter 33 transmits tools constitut- 
ing the coding system used in the picture encoder 32 in 
order for the terminal on the other end to recognize the 
coding system. In Learning Phase described above, 
contacts a and c of a switch are closed and the coding 
system descriptor code used in 1he apparatus is trans- 
mitted to the partner terminal. When the control pro- 
ceeds on to Transmission Phase, contacts b and c of 
the switch are closed and the picture coded data formed 
by the picture encoder 32 is transmitted. 

With this configuration, a software download type 
system can be constructed. Consequently, any new de- 
veloped coding system can be processed. 

Presently, various coding systems have been de- 
veloped in addition to the systems described above, and 
it is difficult to discriminate between them, i.e., it is diffi- 
cult to choose one of these coding systems. These cod- 
ing systems have their respective features, and different 
types of pictures have different appropriate coding sys- 
tems. Also, cameras for inputting pictures may be 
switched, or panned or tilted, or the object itself may be 
changed during communication. In these cases the 
properties of an input picture largely change. Conse- 
quently, it is necessary to switch coding systems during 
communication in order to transmit clearer pictures in a 
narrow band. However, in the conventional picture com- 
munication systems the coding systems cannot be 
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switched during communication. 

SUMMARY OF THE INVENTION 

To simultaneously solve the above conventional 
problems, it is an object of the present invention to pro- 
vide a picture reproducing apparatus, which can recon- 
figure the contents of hardware processing and software 
processing in accordance with a change in a picture re- 
producing algorithm and therefore constantly has an op- 
timum processability for a picture reproducing algo- 
rithm. 

It is another object of the present invention to pro- 
vide a program download system of an encoding/decod- 
ing apparatus by which whether a decoding tool and an 
algorithm normally operate is checked on a receiver side 
(decoder) and a decoding tool and an algorithm not 
found to normally operate by the decoder can be re- 
transmitted from a transmitter side without decreasing 
the operating efficiencies of a communication transmis- 
sion path and a terminal device on the receiver side. 

It is still another object of the present invention to 
provide a picture reproducing apparatus for transmitting 
a coding system descriptor code and selecting an opti- 
mum coding system for each input frame. 

To achieve the above objects, the aspects of the 
present invention are as follows. 

The first aspect of the present invention is a picture 
reproducing apparatus for transmitting coded picture 
data and an algorithm for reproducing (decoding) the 
coded picture data, and reconfiguring a decoding proc- 
essor in accordance with the received algorithm, com- 
prising a program converter for converting a program in 
which the picture reproducing algorithm is described in 
a logical description language into a form which can 
change a configuration of a picture decoder wherein a 
processing system of the picture decoder is reconfig- 
ured in accordance with contents of the program con- 
verted by the program converter. 

The second aspect of the present invention is a pic- 
ture reproducing apparatus for transmitting coded pic- 
ture data and an algorithm for reproducing (decoding) 
the coded picture data, and reconfiguring a decoding 
processor in accordance with the received algorithm, 
comprising a receiver for receiving a program in which 
the picture reproducing algorithm is described in a log- 
ical description language, a program converter for con- 
verting the program into a form which can change a con- 
figuration of a picture decoder, and a controller for con- 
trolling the receiver, the program converter, and the pic- 
ture decoder, wherein the controller causes the program 
converter to convert the program received by the receiv- 
er and reconfigures a processing system of the picture 
decoder in accordance with contents of the converted 
program. 

The third aspect of the present invention is a picture 
reproducing apparatus described in the second aspect, 
wherein the program converter comprises a divider for 
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dividing the picture reproducing algorithm into a first pro- 
gram portion which converts the program described in 
a logical description language into a form which can 
change the configuration ot the picture decoder, and a 
second program portion which is executed by the con- s 
troller in picture reproduction, a first converter for con- 
verting the first program portion into a form which can 
change the configuration of the picture decoder, and a 
second converter for converting the second program 
portion into a form which can be interpreted and execut- io 
ed by the controller. 

The fourth aspect of the present invention is a pic- 
ture reproducing apparatus described in the second or 
third aspect, wherein the picture decoder comprises an 
inverse quantizer for inversely quantizing quantized pic- is 
ture data, contents of the inverse quantization being 
changeable, and an inverse transformer for inversely 
transforming transformed picture data, contents of the 
inverse transformation being changeable, and the con- 
troller changes the contents of the inverse quantization 20 
and the contents of the inverse transformation in accord- 
ance with the contents of the program converted by the 
program converter. 

The fifth aspect of the present invention is a picture 
reproducing apparatus which transmits coded picture 2s 
data and an algorithm for reproducing (decoding) the 
coded picture data, and reconfigures a decoding proc- 
essor in accordance with the received algorithm, and in 
which an encoder transmits to a decoder coded data 
and a tool constituting an algorithm for decoding the 30 
coded data, and the decoder reconfigures the algorithm 
by using the tool and decodes the received coded data 
in a program download manner by using the reconfig- 
ured algorithm, wherein the encoder transmits to the de- 
coder a test routine for performing an operation test of 35 
the algorithm together with the tool, and the decoder 
performs the operation test in accordance with the test 
routine and checks on the basis of the result of the op- 
eration test whether the reconfigured algorithm normally 
operates. 40 

The sixth aspect of the present invention is a picture 
reproducing apparatus described in the fifth aspect, 
wherein the test routine includes a test item for each tool 
constituting the algorithm, and the decoder performs an 
operation test of the algorithm tor each tool in accord- *s 
ance with the test item and checks on the basis of the 
result of the operation test for each tool whether the 
reconfigured algorithm normally operates. 

The seventh aspect of the present invention is a pic- 
ture reproducing apparatus described in the fifth aspect, so 
wherein the test routine is provided for each tool, and 
the decoder performs an operation test for each tool in 
accordance with the test routine provided for the tool 
and checks on the basis of the result of the operation 
test for each tool whether the reconfigured algorithm ss 
normally operates. 

The eighth aspect of the present invention is a pic- 
ture reproducing apparatus described in the fifth aspect, 



wherein the decoder requests the encoder to retransmit 
a tool which constitutes the algorithm and is not found 
to normally operate, and on the basis of the request the 
encoder retransmits to the decoderthe tool not found to 
normally operate by the decoder. 

The ninth aspect ot the present invention is a picture 
reproducing apparatus described in the fifth aspect, 
wherein when the decoder determines that a tool nor- 
mally operates, the decoder immediately executes a 
procedure of decoding coded data to be processed by 
the tool. 

The tenth aspect of the present invention is^pjclure 
reproducing apparatus for transmitting coded picture 
data and an algorithm for reproducing (decoding) the 
coded picture data, and reconfiguring a decoding proc- 
essor in accordance with the received algorithm, com- 
prising picture encoding means for encoding an input 
picture, coding system descriptor code transmitting 
means for transmitting to a partner terminal a coding 
system descriptor code describing a coding system 
used by the picture encoding means.-axommunication 
state controller for managing a state of communication 
and, in accordance with the state of communication, se- 
lecting one of transmission of the picture coded data and 
transmission of the coding system descriptor code, a 
plurality of picture encoding means for encoding an in- 
put picture by using different coding systems, and cod- 
ing system selecting means for determining a coding 
system to be used, wherein an appropriate picture cod- 
ing system is selected for each frame of an input picture 
and, when coded data is transmitted to the partner ter- 
minal, the partner terminal is informed of a coding sys- 
tem by which each frame of data is encoded, thereby 
dynamically switching picture coding systems during 
communication. 

The eleventh aspect of the present invention is a 
picture reproducing apparatus described in the tenth as- 
pect, wherein the number of coding system descriptor 
codes transmitted immediately after communication is 
started is set in accordance with a memory capacity of 
the partner terminal and, when a user wishes to use a 
coding system other than the previously-transmitted 
coding systems, one of the already transmitted coding 
systems is made invalid and the coding system descrip- 
tor code to be used instead is transmitted and made val- 
id. 

The twelfth aspect of the present invention is a pic- 
ture reproducing apparatus described in the tenth as- 
pect, wherein a plurality of coding system descriptor 
codes received from the partner terminal when commu- 
nication is started or while communication is being per- 
formed are correctly recognized, and a coding system 
by which received picture coded data is encoded is 
checked for each frame, thereby dynamically switching 
picture coded data decoding means during communica- 
tion. 

The thirteenth aspect of the present invention is a 
picture reproducing apparatus described in the eleventh 
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aspect, wherein a plurality of coding system descriptor 
codes received from the partner terminal when commu- 
nication is started or while communication is being per- 
formed are correctly recognized, and a coding system 
by which received picture coded data is encoded is s 
checked for each frame/thereby dynamically switching 
picture coded data decoding-means during communica- 
tion. 

The fourteenth aspect of the present invention is a 
picture reproducing apparatus described in the tenth as- io 
pect, wherein the coding system selecting means com- 
prises picture encoding means whose number equals 
the number of coding systems for performing picture en- 
coding, picture decoding means whose number equals 
the number of coding systems, squared error calculating is 
means whose number equals the number of coding sys- 
tems, each of the squafed error calculating means cal- 
culating a squared error between a picture encoded/de- 
coded result and an input picture as an original picture, 
and squared error result comparing means for checking 20 
which coding system has a smallest squared error, and, 
in order to select a coding system for performing coding, 
calculates a squared error in each coding system for 
each frame and selects a coding system with a smallest 
squared error. 2s 

The fifteenth aspect of the present invention is a pic- 
ture reproducing apparatus described in the eleventh 
aspect, wherein the coding system selecting means 
comprises picture encoding means whose number 
equals the number of coding systems for performing pic- 30 
ture coding, picture decoding means whose number 
equals the number of coding systems, squared error cal- 
culating means whose number equals the number of 
coding systems, each of the squared error calculating 
means calculating a squared error between a picture en- 3S 
coded/decoded result and an input picture as an original 
picture, and squared error result comparing means for 
checking which coding system has a smallest squared 
error, and, in order to select a coding system for per- 
forming coding, calculates a squared error in each cod- 40 
ing system for each frame and selects a coding system 
with a smallest squared error. 

The sixteenth aspect of the present invention is a 
picture reproducing apparatus described in the twelfth 
aspect, wherein the coding system selecting means "5 
comprises picture encoding means whose number 
equals the number of coding systems for performing pic- 
ture coding, picture decoding means whose number 
equals the number of coding systems, squared error cal- 
culating means whose number equals the number of so 
coding systems, each of the squared error calculating 
means calculating a squared error between a picture en- 
coded/decoded result and an input picture as an original 
picture, and squared error result comparing means for 
checking which coding system has a smallest squared ss 
error, and, in order to select a coding system for per- 
forming coding, calculates a squared error in each cod- 
ing system for each frame and selects a coding system 



with a smallest squared error. 

The seventeenth aspect of the present invent ion is 
a picture reproducing apparatus described in the thir- 
teenth aspect, wherein the coding system selecting 
means comprises picture encoding means whose 
number equals the number of coding systems for per- 
form ingipicture coding, picture decoding:means whose 
number equals the number of coding systems, squared 
error calculating means whose number equals the 
number of coding systems, each of the squared error 
calculating means calculating a squared error between 
a picture encoded/decoded result and an input picture 
as an original picture, and squared error result compar- 
ing means for checking which coding system has a 
smallest squared error, and, in order to select a coding 
system for performing coding, calculates a squared er- 
ror in each coding system for each frame and selects a 
coding system with a smallest squared error. 

The eighteenth aspect of the present invention is a 
picture reproducing apparatus described in any one of 
the tenth to seventeenth aspects, wherein the coding 
system selecting means comprises picture adaptive 
coding system selecting means for selecting an appro- 
priate coding system in accordance with an input pic- 
ture, and determines a coding system for performing 
coding by checking, for each frame, which coding sys- 
tem is appropriate to encode a picture. 

In the above picture coding apparatus, the plurality 
of picture encoding means form picture coded data by 
using different picture coding systems, the coding sys- 
tem selecting means selects a coding system for each 
frame, the picture decoding means decode the data 
formed by picture encoding, each squared error calcu- 
lating means calculates a squared error between an in- 
put picture and a picture formed by encoding/decoding, 
the squared error determining means determines which 
coding system has the smallest squared error in encod- 
ing/decoding, and the picture checking means checks 
a type of input picture and determines which coding sys- 
tem is optimum tor the picture. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 A is a view showing the lormat of a data se- 
quence o1 coded data obtained by encoding (com- 
pressing) picture data in accordance with H. 261: 
Fig. IB is a view showing the format of a data se- 
quence of coded data obtained by encoding (com- 
pressing) picture data by using a coding system 
with a flexible algorithm such as MPEG4; 
Fig. 2 is a block diagram showing the configuration 
of an encoder based on H. 261 ; 
Fig. 3 is a block diagram showing the configuration 
of a decoder based on H. 261 ; 
Fig. 4 is a block diagram showing the configuration 
of an encoder including a plurality of algorithms; 
Fig. 5 is a block diagram showing the configuration 
of a decoder implemented by dedicated hardware 
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and software; 

Fig. 6 is a block diagram showing the configuration 
of a decoder implemented by a gen era I -purpose 
arithmetic processor and a compiler; 
Fig.7 is a flow chart for explaining an operation al- 
gorithm of the decoder; 

Fig.8 is a view showing the components of MSDL; 
Fig. 9 isa block diagram showing a conventional pic- 
ture encoding apparatus; 

Fig. 1 0 is a block diagram showing the principle of a 
picture reproducing apparatus according to the 
present invention; 

Fig. 1 1 is a block diagram showing one embodiment 
of the picture reproducing apparatus according to 
the present invention; 

Fig. 12 is a view for explaining the lormat of a trans- 
mission data sequence according to a program 
download method of the second embodiment of the 
present invention; 

Fig. 1 3 is a flow chart for explaining an operation al- 
gorithm of a decoder according to the program 
download method of the second embodiment of the 
present invention; 

Fig. 1 4 is a view for explaining the format of a trans- 
mission data sequence according to a program 
download method of the third embodiment of the 
present invention; 

Fig. 1 5 is a flow chart for explaining an operation al- 
gorithm of a decoder according to the program 
download method of the third embodiment of the 
present invention; 

Fig. 16 is a block diagram of a picture encoding ap- 
paratus of the present invention; 
Figs. 1 7 shows data sequences transmitted f rom the 
picture encoding apparatus of the present inven- 
tion: 

Figs. 18 shows transmission data sequences when 
coding system assignment is changed; 
Fig. 19 is a block diagram of a coding system selec- 
tion method using squared error calculations; and 
Fig. 20 is a flow chart of a picture adaptive coding 
system selection method. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

Embodiments of the present invention will be de- 
scribed below with reference to the accompanying 
drawings. 

(First Embodiment) 

Fig. 10 is a block diagram showing the principle of a 
picture reproducing apparatus according to the present 
invention. This picture reproducing apparatus reproduc- £ 
es moving pictures and comprises a program converter 
111 and a moving picture decoder 112. A program in 
which a moving picture reproducing algorithm is de- 



scribed in a logical description language is input to the 
program converter 111 where the program is converted 
into a form which can change the configuration of the 
moving picture decoder 112. In accordance with the 
5 converted program, the moving picture decoder 112 
reconfigures the processing system of hardware 
processing or software processing, decodes coded 
moving picture data, and reproduces the moving pic- 
ture. 

io As described above, before moving .picture.data is 
reproduced a moving picture reproducing algorithm is 
received and the moving picture decoder is reconfig- 
ured. Consequently, a plurality of moving picture repro- 
ducing algorithms can be executed in a single moving 
is picture reproducing apparatus. Also, since the moving 
picture decoder is reconfigured in accordance with a 
moving picture reproducing algorithm, a moving picture 
reproducibility equivalent to that of a dedicated moving 
picture reproducing apparatus can be obtained. 

zo Fig. 1 1 is a block diagram showing one embodiment * 
of the picture reproducing apparatus according to the 
present invention. This picture reproducing apparatus 
comprises a receiver 116, a switching means 117, a pro- 
gram converter 111, a moving picture decoder 112 

2S whose processing system can be reconfigured, and a 
controller 118. The receiver 116 downloads a program 
for reconfiguring the moving picture decoder and coded 
moving picture data from a network 115. The switching 
means 117 switches transmission destinations in ac- 

so cordance with whether received data is a program or 
moving picture data. The program converter 111 con- 
verts the program into a form which can change the con- 
figuration of the moving picture decoder. The controller 
118 controls the receiver 116. the switching means 117, 

35 the program converter 111 , and the moving picture de- 
coder 112. 

The program converter 111 includes a divider 119, 
a first converter 120, and a second converter 121 . The 
divider 119 divides the program into a first program por- 

*o tion to be converted into a form which can change the 
configuration of the moving picture decoder and a sec- 
ond program portion which is executed by the controller 
during reproduction of moving pictures. The first con- 
verter 120 converts the first program portion into a form 

*5 capable of changing the configuration of the moving pic- 
ture decoder. The second converter 1 21 converts the 
second program portion into a form which can be inter- 
preted and executed by the controller. The moving pic- 
ture decoder 112 includes an inverse quantizer 122.and 

>o an inverse transformer 1 23. The inverse quantizer 1 22 
inversely quantizes quantized moving picture data, and 
the contents of the inverse quantization can be 
changed. The inverse transformer 123 inversely trans- 
forms transformed moving picture data, and the con- 

s tents of the inverse transform can be changed. Since 
the processing contents of the inverse quantizer 122 
and the inverse transformer 123 must be rewritable, 
these components need to be programmable elements 
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such as FPGAs. 

This moving picture reproducing apparatus re- 
ceives a program describing a picture reproducing algo- 
rithm and coded moving picture data from the network, 
automatically reconfigures the moving picture decoder 5 
112 in accordance with the program, and reproduces 
moving pictures by using the algorithm of the program. 

A procedure by which the moving picture decoder 
is reconfigured is as follows. The network 115 inputs a 
moving picture reproducing program and coded moving io 
picture data to the moving picture reproducing appara- 
tus. First, the receiver 116 receives a program of a mov- 
ing picture reproducing algorithm described in a logical 
description language from the network 115. The control- 
ler 1 18 controls the switching means 1 1 7 to transmit the is 
received data to the program converter 111. The divider 
119 of the program converter 111 divides the program 
into a first program portion to be processed by hardware 
and a second program portion to be processed by soft- 
ware. The software processing during reproduction of 20 
moving pictures is performed by a processor in the con- 
troller 118. The portion to be processed by hardware is 
transferred to the first converter 1 20 and converted into 
a form capable of rewriting FPGA. Thereafter, the con- 
troller 118 rewrites the inverse quantizer 122 and the 2s 
inverse transformer 123. The portion to be processed 
by software is transferred to the second converter 121, 
converted into a form capable of being processed by the 
processor, and transferred to the controller 1 1 8. 

As described above, since the inverse quantizer $0 
122 and the inverse transformer 123 are formed by el- 
ements such as FPGAs whose processing programs 
are changeable, the hardware processing in the moving 
picture decoder 112 can be easily reconfigured. Addi- 
tionally, since the moving picture decoder 112 whose 35 
hardware is so reconfigured as to have an optimum 
processability is used, a moving picture reproducibility 
equivalent to that of a dedicated moving picture decoder 
can be obtained. 

In reproduction of moving pictures, the receiver 116 *o 
receives moving picture data from the network 115. The 
controller 1 1 8 controls the switching means 1 1 7 to trans- 
mit the moving picture data to the moving picture decod- 
er 112. The moving picture data is processed by the in- 
verse quantizer 122 and the inverse transformer 1 23 in *s 
the moving picture decoder 112 and output as decoded 
data. 

In the first embodiment of the present invention de- 
scribed above, the processing system of the picture de- 
coder is reconfigured in accordance with the contents so 
of a program converted by the program converter. 
Therefore, picture data to be reproduced by different al- 
gorithms can be processed. Also, a picture reproduci- 
bility equivalent to that of a dedicated picture reproduc- 
ing apparatus can be obtained. In addition, a program ss 
describing a picture reproducing algorithm is input to- 
gether with picture data from the network, and the con- 
troller reconfigures the picture decoder in accordance 



with the program so that the decoder has an optimum 
processability. Accordingly, picture data to be repro- 
duced by different algorithms can be automatically proc- 
essed. 

Furthermore, a program describing a picture repro- 
ducing algorithm is divided into a first program portion 
to be converted into a form which can change the con- 
figuration of the picture decoder and a second program 
portion to be executed by the controller during picture 
reproduction. Consequently, the moving picture decod- 
er can be so reconfigured as to have an opt i mum proc- 
essabilrty. Therefore, a picture reproducibility equivalent 
to that of a dedicated picture reproducing apparatus can 
be obtained. 

<Second Embodiment) 

Fig. 1 2 is a view showing an example of a coded da- 
ta sequence based on a flexible coding system such as 
MPEG4 according to a program download system of the 
present invention. The coded data sequence shown in 
Fig. 12 consists of a test routine which is a program rou- 
tine for performing an operation test of a decoding algo- 
rithm, tool information following the test routine 231, 
such as a motion compensation tool 232a, an inverse 
transform tool 232b, and an inverse quantization tool 
232c, and picture information data such as motion vec- 
tor information 233a, a transform coefficient 233b, and 
a quantization step 233c. The test routine 231 contains 
test items for the individual tools. Operation tests are 
performed for the tool information 232a to 232c consti- 
tuting the decoding algorithm on the basis of these test 
items. Consequently, the operation tests of the decoding 
algorithm constituted by these tool information 232a to 
232c can be performed on the decoder side. 

Fig. 1 3 is a flow chart for explaining an algorithm for 
decoding coded data having the format shown in Fig. 
12. When all of the tool information 232a to 232c con- 
stituting the decoding algorithm are completely re- 
ceived, operation tests are sequentially performed for 
these tools 232a to 232c on the basis of the test routine 
231 , and the result is checked each time the test is per- 
formed. Details of this procedure will be described be- 
low. 

First, a decoding algorithm test is executed on the 
basis of the test routine 231 contained in the received 
coded data (step S21). The flow then advances to an 
operation test of each tool. An operation test of the tool 
232a is executed and the result is checked (step S22). 
If it is determined that the tool 232a normally operates 
(Yes in step S22), the motion vector information 233a is 
decoded by using the tool 232a (step S24), and the flow 
advances to an operation test of the tool 232b. If it is 
determined that the operation of the tool 232a is abnor- 
mal (No in step S22), a request to retransmit the tool 
232a is output to the transmission side, and the flow im- 
mediately advances to the operation test of the tool 
232b. 
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The operation test of the tool 232b is-executedand 
the result is checked (step S25). If it is determined that 
the tool 232b normally operates (Yes in step S25), the 
transform coefficient information 233b is decoded by us- 
ingthe tool 232b (step S27), and the flow advances to s 
an operation tesrof the tool 232c. If it is determined that 
the operation of the tool 232b is abnormal (No in step 
S25) : a request to retransmit the tool 232b is output to 
the transmission side, and the flow immediately advanc- 
es to the_operation::test of the tool 232c. to 

The operation test of the tool 232c is executed and 
the result is checked (step S28). If it is determined that 
the tool 232c normally operates (Yes in step S28), the 
quantization step information 233c is decoded by using 
the tool 232c (step S30), and the flow waits until the next is 
coded information data is received. If it is determined 
that the operation of the tool 232c is abnormal (No in 
step S28), a request to retransmit the tool 232c is output 
to the transmission side, and the flow waits until the next 
coded data is received. 20 

F01 a tool found to abnormally operate by an oper- 
ation test of the tool, a request to retransmit the tool is 
output to the encoder (step S23, S26, or S29). Thereaf- 
ter, the flow immediately advances to an operation test 
ol the next tool. If the operation test is normally complet- 
ed, decoding of picture information data to be processed 
by that tool is started (step S24, S27, or S30). 

For example, if the result ol the operation test of the 
tool 232a is found to be abnormal, the decoder requests 
the encoder to retransmit the tool 232a (step S23). While 
the encoder is executing a procedure ol retransmitting 
the tool 232a. the decoder executes the operation test 
of the tool 232b and checks the test result (step S25). 

In contrast, in the prior art shown in Fig. 7. when the 
decoder detects an error in a transmission block of the 
tool 1, the encoder executes a procedure of retransmit- 
ting this transmission block (step S2), and the next block 
is not transmitted until this retransmission procedure is 
completed. Therefore, when a transmission path in 
which a large number of errors occur is used, the 
processing efficiency of the decoder decreases. In ad- 
dition, the start of decoding of coded data received by 
the decoder is delayed. 

In the second embodiment of the present invention 
as described above, a test routine for performing oper- 
ation tests of tools is transmitted together with the tools. 
This allows a decoder to perform the operation tests of 
these tools. Furthermore, while an encoder is executing 
a procedure of retransmitting a tool found to abnormally 
operate, a decoder executes an operation test of the so 
next tool. Consequently, the operating efficiencies of the 
decoder and the communication transmission path can 
be raised. 
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(Third Embodiment) 

Fig. 1 4 is a view showing another example of the for- 
mat of a coded data sequence based on a flexible cod- 



ss 



ing system such as MPEG4 according to the program 
download method of the present invention. Tools 242a, 
242b, and 242c have test routines 241a, 241b, and 
241c, respectively, and so a decoder on the reception 
side can independently perform operation tests of these 
tools. 

Fig. 1 5 is a flow chart for explaining an algorithm for 
decoding coded data having the format shown in Fig. 
14. When tool information contained in the coded data 
is received, an operation test of the tool is immediately 
executed on the basis of the test routine which the re- 
ceived tool information includes, and the operation of 
the tool is checked (step S31 , S35, or S39). If it is de- 
termined that the operation is abnormal, a request to re- 
transmit the tool is output to the encoder (step S33, S37, 
or S41 ). If the operation is found to be normal, data to 
be processed by the tool is decoded by using the tool 
(step S34, S38, or S42). That is, in this embodiment the 
operation test is executed in units of tools and the re- 
transmission procedure is executed in units of tools. 

More specifically, an operation test of the tool 242a 
contained in the received coded data is executed, and 
the result of the test is checked (step S31). If it is deter- 
mined that the tool normally operates, decoding is per- 
formed by using the tool (step S34), and the flow ad- 
vances to an operation test of the tool 242b. If the op- 
eration is found to be abnormal, a request to retransmit 
the tool 242a is output to the transmission side (step 
S33), and the flow immediately advances to the opera- 
tion test of the tool 242b. Operation tests of the tools 
242b and 242c are similarly performed and, in accord- 
ance with the test results, a request to retransmit a tool 
is output to the transmission side. A normally operating 
tool is immediately used in decoding. After the operation 
tests of a series of tools and decoding using these tools 
are executed s operation tests of the retransmitted tools 
are performed. A retransmission request is again output 
for a tool found to abnormally operate by this operation 
test. A tool found to normally operate is used to execute 
decoding. The decoder repeatedly requests the encod- 
er to retransmit a tool as long as there is a tool found to 
abnormally operate by the operation test. While the en- 
coder is executing the retransmission procedure, the 
decoder executes decoding using the tool found to nor- 
mally operate. 

In the third embodiment as described above, as in 
the second embodiment, while the encoder is executing 
a retransmission procedure for a tool whose operation 
is found to be abnormal, the decoder can execute an 
operation test of the next tool. Accordingly, the operating 
efficiencies of the decoder and the communication 
transmission path can be raised. In addition, since each 
tool has a test routine, an operation test can be per- 
formed for each tool. 

As is apparent from the above description, the sec- 
ond and third embodiments of the present invention 
achieve the following effects. 

When an encoder transmits tools constituting an al- 
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gorithm as a means for decoding coded data, a test rou- 
tine for perform ing~an~operation test of the algorithm is 
simultaneously transmitted. This allows a decoder to 
check whether the received decoding algorithm normal- 
ly operates. 5 

The test routine of the decoding algorithm includes 
test items of tooJs. consljtutjng-.the decoding aJgorithm. 
Therefore, the decoder can check for each tool consti- 
tuting the decoding algorithm whether the tool can nor- 
mally operate. 10 

Also, sincea test routine is provided for each tool 
constituting the decoding algorithm, an operation test 
can be independently performed for each tool. This al- 
lows the decoder to independently check for each re- 
ceived tool whether the tool normally operates. is 

The decoder can obtain a normal tool by requesting 
retransmission of a tool found to abnormally operate. 
Accordingly, the decoder can perform decoding free of 
errors by using this tool. 

Furthermore, when a tool is found to normally op- 20 
erate by an operation test performed by the decoder, 
decoding of coded data (picture information) to be proc- 
essed by the tool is immediately started. Consequently, 
the operating efficiencies of the decoder and the com- 
munication transmission path can be raised. 2s 

(Fourth Embodiment) 

Fig. 16 is a block diagram of a picture encoding ap- 
paratus of the present invention. A communication state so 
controller 71 manages the states of encoders and con- 
trols a switch SWT for switching transmission of picture 
coded data and transmission of a coding system de- 
scriptor code. A picture encoder 72a using a coding sys- 
tem A encodes an input picture by using an encoding 35 
system indicated by the coding system descriptor code 
A. A picture encoder 72b using a coding system B en- 
codes an input picture by using an encoding system in- 
dicated by the coding system descriptor code B. A pic- 
ture encoder 72c using a coding system C encodes an *o 
input picture by using an encoding system indicated by 
the coding system descriptor code C. A picture encoder 
722 using a coding system 2 encodes an input picture 
by using an encoding system indicated by the coding 
system descriptor code 2. A coding system selector 73 
is connected to the communication state controller 71. 
The coding system selector 73 selects a coding system 
for each frame and controls a switch SW2 for selecting 
output data to be transmitted from the picture encoders. 
A coding system descriptor code transmitter 74 is con- so 
nected to the coding system selector 73 and transmits 
a coding system descriptor code. To transmit a coding 
system descriptor code, contacts a and c of the switch 
SW1 are closed. To transmit picture coded data, con- 
tacts b and c of the switch SW1 are closed. ss 

Fig. 17 shows examples of data sequences trans- 
mitted from the picture encoding apparatus of the 
present invention. In Configuration Phase, limiting items 



such as the memory capacity of the partner terminal are 
checked: In Learning Phase, the number of coding sys- 
tem descriptor codes within the range of the memory 
capacity of the partner terminal are transmitted. In prac- 
tice, tools constituting the coding system are transmitted 
as described above. More specifically, the contacts a 
andicofthe-switch SW1 in Fig, I6_a re cJosed,.and coding 
system numbers are added to coding system descriptor 
codes to be transmitted so that the coding system num- 
bers are in a one-to-one correspondence with the cod- 
ing system descriptor codes. One practical example will 
be described below with reference to Fig.17(a). Coding 
system number 1 is first transmitted, and tools consti- 
tuting the coding system A are subsequently transmit- 
ted. As a consequence, the coding system A is assigned 
to coding system number 1. When the transmission of 
the coding system A is completed, coding system 
number 2 is transmitted and tools constituting the coding 
system B are subsequently transmitted. Likewise, cod- 
ing system number 3 and tools constituting the coding 
system C are transmitted. By a series of these opera- 
tions, the coding systems A, B, and C are assigned to 
coding system numbers 1 , 2 : and 3, respectively. 

Fig. 1 7(b) shows an example of a transmission data 
sequence in Transmission Phase for transmitting coded 
data. In Transmission Phase, the contacts b and c of the 
switch SW1 in Fig. 16 are closed, and a coding system 
number is added to the leading position of each frame. 
This number is made correspond to the coding system 
in Learning Phase. This number clearly indicates the 
coding system by which the coded data sequence fol- 
lowing the number is to be decoded. A practical opera- 
tion will be described with reference to Fig. 17(b). If it is 
determined that a f rame n is to be encoded by the coding 
system B, coding system number 2 to which the coding 
system B is assigned is transmitted. Subsequently, cod- 
ed data obtained by encoding the frame n by using the 
coding system B is transmitted. If it is determined that a 
next frame n+1 is to be encoded by the coding system 
A, coding system number 1 to which the coding system 
A is assigned is transmitted. Subsequently, coded data 
obtained by encoding the frame n+1 by using the coding 
system A is transmitted. If it is determined that a next 
frame n+2 is to be encoded by the coding system B, 
coding system number 2 is transmitted and coded data 
obtained by encoding the Irame n+2 by using the coding 
system B is transmitted. 

Fig. 18 shows examples of transmission data se- 
quences when coding system assignment is changed. 
Fig. 18 shows a case where the number of coding sys- 
tems to be transmitted is limited to 3 due to the memory 
capacity of the partner terminal. As in the case of Fig. 
17, three coding system descriptor codes of the coding 
systems A, B : and C are transmitted in Learning Phase 
when a communication line is initially connected, and 
the coding systems A, B, and C are assigned to coding 
system numbers 1 , 2, and 3, respectively. Assume that 
the user wishes to use the coding system 2 thereafter. 
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If the coding system recognition capacity of the partner 
terminal is large enough, the coding system 2 is as- 
signed to coding system number 4 and transmitted 
However, if the partner terminal can recognize only 
three coding systems due to the influence of the limited 
memory capacity, it is necessary to invalidate one of the 
coding systems previously transmitted. Assume that the 
coding system C is to be made invalid. Since the coding 
system C is assigned to coding system number 3, it is 
only necessary to assign the coding system 2 to coding 
system number 3 and transmit the coding system. That 
is, as shown in Fig. 18(b), coding system number 3 is 
transmitted and subsequently tools constituting the cod- 
ing system Z are transmitted. 

Fig.19 is a block diagram showing a squared error 
selection method as one coding system selection meth- 
od. In Fig. 19, picture encoders 61 a to 61 z using the re- 
spective coding systems encode an input picture by us- 
ing encoding systems indicated by the respective cod- 
ing systems. Picture decoders 62a to 62z using the re- 
spective coding systems are connected to the picture 
encoders 61a to 61z, respectively, and decode the cod- 
ed picture data by using decoding systems indicated by 
the respective coding systems. Squared error calcula- 
tors 63a to 63z are connected to the picture decoders 
62a to 62z, respectively, and calculate squared errors 
between the decoded pictures encoded/decoded by the 
respective coding systems and the input picture. A 
squared error result comparator 64 is connected to the 
squared error calculators 63a to 63z and compares the 
squared error evaluations of the results of encoding/de- 
coding based on the respective coding systems. 

In this method, a suitable picture coding system is 
selected for each frame. That is, encoding and decoding 
are performed by using all coding systems, squared er- 
rors between the decoded pictures and the input picture 
are calculated, and a coding system having the smallest 
squared error is selected. A squared error is obtained 
by calculating the difference of an encoded/decoded 
picture from an input picture for each pixel and accumu- 
lating the differences. That is, an encoded/decoded pic- 
ture having a good squared error accumulation result is 
close to an input picture. Therefore, when a coding sys- 
tem having the smallest squared error is chosen for 
each frame, pictures closest to the respective input pic- 
tures can be transmitted to the partner terminal for all 
frames. 

Fig. 20 is a flow chart of a picture adaptive coding 
system selection method as a second coding system se- 
lection method. In this picture adaptive coding system 
selection method, whether an input picture is an appro- 
priate one to be encoded by a certain coding system is 
checked for each frame. This determination is done by 
performing a certain calculation for an input picture and 
checking on the basis of the calculation result whether 
the picture is an appropriate one to be encoded by using 
a coding system of interest. 

As the calculation used in determination of whether 
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an input picture is an appropriate picture, a DCT calcu- 
lation is performed for a picture coding system effective 
when a picture has a larger number of fine patterns. That 
is, an adaptive coding determination calculation is per- 
s formed in which effectiveness is determined when the 
accumulation of coefficients of nth order or more is a 
predetermined value or larger. For a picture coding sys- 
tem effective when a picture has a large motion compo- 
nent, an adaptive coding determination calculation is 
io performed in which effectiveness is determined when 
the accumulation of differences between frames in each 
pixel is a predetermined amount or more. 

In the flow chart of Fig.20 t one of three coding sys- 
tems, i.e., the coding systems A, B, and C, is selected. 
First, a picture is input and whether the input picture is 
an appropriate one to be encoded by using the coding 
system A is checked. That is, an adaptive coding deter- 
mination calculation of the coding system A is per- 
formed, and whether the calculation result satisfies the 
criterion for the coding system A is checked (step S51). ' 
If this input picture satisfies the criterion of whether the 
picture is an appropriate one to be encoded by the cod- 
ing system A, it is determined that the input picture is 
encoded by the coding system A (step S52). If the input 
picture does not satisfy the criterion, whether the input 
picture is an appropriate one to be encoded by the cod- 
ing system B is checked. That is, an adaptive coding 
determination calculation of the coding system B is per- 
formed, and whether the calculation result satisfies the 
criterion for the coding system B is checked (step S53). 
If this input picture satisfies the criterion of whether the 
picture is an appropriate one to be encoded by the cod- 
ing system B, it is determined that the input picture is 
encoded by the coding system B (step S54). If the input 
picture does not satisfy the criterion, whether the input 
picture is an appropriate one to be encoded by the cod- 
ing system C is checked. That is, an adaptive coding 
determination calculation of the coding system C is per- 
formed, and whether the calculation result satisfies the 
criterion for the coding system C is checked (step S55). 
If this input picture satisfies the criterion of whether the 
picture is an appropriate one to be encoded by the cod- 
ing system C, it is determined that the input picture is 
encoded by the coding system C (step S56). If the input 
picture does not satisfy the criterion, it is determined that 
the input picture is encoded by a previously determined 
default coding system (step S57). 

The fourth embodiment of the present invention de- 
scribed above can achieve the following effects. 

The apparatus comprises a plurality of picture en- 
coding means for encoding an input picture by using dif- 
ferent coding systems and a coding system selecting 
means for selecting a coding system to be used. A prop- 
er picture coding system is selected for each frame of 
55 an input picture, and the partner terminal is informed of 
the coding system by which each frame of data is en- 
coded. Therefore, picture coding systems can be dy- 
namically switched during communication. 
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The number of coding systems to be transmitted im- 
mediately after communication is started is set in ac- 
cordance the memory capacity of the partner terminal. 
It the user wants to use a coding system other than the 
previously transmitted coding systems, one of the al- 
ready transmitted coding systems is made invalid, and 
the coding system of interest is transmitted in place of 
the nullified one and made valid. This makes the coding 
system other than the previously transmitted coding 
systems usable during communication. 

A plurality of coding system descriptor codes re- 
ceived from the partner terminal when communication 
is started or during communication are correctly recog- 
nized, and the coding system by which the received pic- 
ture coded data is encoded is checked for each frame. 
Accordingly, means for decoding the picture coded data 
can be dynamically switched during communication. 

The coding system selecting means comprises a 
plurality of picture decoding means which perform de- 
coding by the same coding systems as picture encod- 
ing, and the number of which equals the number of the 
coding systems, a plurality of squared error calculating 
means which calculate squared errors between picture 
encoded/decoded results and an input picture, and the 
number of which equals the number of the coding sys- 
tems, and a squared error determining means for deter- 
mining which coding system has the smallest squared 
error. In selecting a coding system by which encoding 
is performed, squared errors of the individual coding 
systems are calculated for each frame. Consequently, 
a coding system having the smallest squared error can 
be chosen for each frame. 

A coding system selecting means comprises a pic- 
ture adaptive coding system selecting means which se- 
lects an appropriate coding system in accordance with 
an input picture. A coding system by which a picture is 
appropriately encoded is determined for each frame. 
Accordingly, an appropriate coding system can be de- 
termined when encoding is performed. 



10 



is 



20 



25 



30 



35 



4. 



40 



A picture reproducing apparatus for transmitting 
coded picture data and an algorithm for reproducing 
(decoding) the coded picture data, and reconfigur- 
ing a decoding processor in accordance with the re- 
ceived algorithm, comprising; 

a receiver for receiving a program in which the 
picture reproducing algorithm is described in a 
logical description language; 
a program converter for converting the program 
into a form which can change"a"configuration of 
a picture decoder; and 

a controller for controlling said receiver, said 
program converter, and said picture decoder, 
wherein said controller causes said program 
converter to convert the program received by 
said receiver and reconfigures a processing 
system of said picture decoder in accordance 
with contents of the converted program. 

The apparatus according to claim 2, wherein said 
program converter comprises: 

a divider for dividing the picture reproducing al- 
gorithm into a first program portion which con- 
verts the program described in a logical de- 
scription language into a form which can 
change the configuration of said picture decod- 
er, and a second program portion which is ex- 
ecuted by said controller in picture reproduc- 
tion; 

a first converter for converting the first program 
portion into a form which can change the con- 
figuration of said picture decoder; and 
a second converter for converting the second 
program portion into a form which can be inter- 
preted and executed by said controller. 

The apparatus according to claim 2 or 3, wherein 
said picture decoder comprises: 



Claims 



A picture reproducing apparatus for transmitting 
coded picture data and an algorithm for reproducing 
(decoding) the coded picture data, and reconfigur- 
ing a decoding processor in accordance with the re- 
ceived algorithm, comprising: 

a program converter for converting a program 
in which the picture reproducing algorithm is 
described in a logical description language into 
a form which can change a configuration of a 
picture decoder, 

wherein a processing system of said picture de- 
coder is reconfigured in accordance with con- 
tents of the program converted by said program 
converter. 
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an inverse quantizer for inversely quantizing 
quantized picture data, contents of the inverse 
quantization being changeable; and 
an inverse transformer for inversely transform- 
ing transformed picture data, contents of the in- 
verse transformation being changeable, and 
said controller changes the contents of the in- 
verse quantization and the contents of the in- 
verse transformation in accordance with the 
contents of the program converted by said pro- 
gram converter. 

A picture reproducing apparatus which transmits 
coded picture data and an algorithm for reproducing 
(decoding) the coded picture data, and reconfigures 
a decoding processor in accordance with the re- 
ceived algorithm, and in which an encoder transmits 
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to a decoder coded information and a tool constitut- 
ing an algorithm for decoding the coded informa- 
tion, and said decoder reconfigures the algorithm 
by using the tool and decodes the received coded 
information in a program download manner by using s 
the reconfigured algorithm, 

wherein said encoder transmits to said decod- 
er a test routine for performing an operation test of 
thaalgorithm together with the tool, and said decod- 
er-performs the operation test in accordance with io 
the test routine and checks on the basis of the result 
of the operation test whether the reconfigured algo- 
rithm normally operates. 

The apparatus according to claim 5, wherein the is 
test routine includes a test item for each tool con- 
stituting the algorithm, and said decoder performs 
an operation test of the algorithm for each tool in 
accordance with the test item and checks on the ba- 
sis of the result of the operation test for each tool 20 
whether the reconfigured algorithm normally oper- 
ates. 

The apparatus according to claim 5, wherein the 
test routine is provided for each tool, and said de- 2s 
coder performs an operation test for each tool in ac- 
cordance with the test routine provided for the tool 
and checks on the basis of the result of the opera- 
tion test for each tool whether the reconfigured al- 
gorithm normally operates. 30 

The apparatus according to claim 5, wherein said 
decoder requests said encoder to retransmit a tool 
which constitutes the algorithm and is not found to 
normally operate, and on the basis of the request 35 
said encoder retransmits to said decoder the tool 
not found to normally operate by said decoder. 

The apparatus according to claim 5, wherein when 
said decoder determines that a tool normally oper- 40 
ales, said decoder immediately executes a proce- 
dure of decoding coded data to be processed by the 
tool. 

A picture reproducing apparatus for transmitting *s 
coded picture data and an algorithm for reproducing 
(decoding) the coded picture data, and reconfigur- 
ing a decoding processor in accordance with the re- 
ceived algorithm, comprising: 

so 

picture encoding means for encoding an input 
picture; 

coding system descriptor code transmitting 
means for transmitting to a partner terminal a 
coding system descriptor code describing a ss 
coding system used by said picture encoding 
means; 

a communication state controller for managing 



a state of communication and, in accordance 
with the state of communication, selecting one 
of transmission of the picture coded data and 
transmission of the coding system descriptor 
code; 

a plurality of picture encoding means for encod- 
ing an input picture by using different coding 
systems; and 

coding system selecting means for determining 
a coding system to be used, 
wherein an appropriate picture coding system 
is selected for each frame of an input picture 
and, when picture coded data is transmitted to 
the partner terminal, the partner terminal is in- 
formed of a coding system by which each frame 
of data is encoded, thereby dynamically switch- 
ing picture coding systems during communica- 
tion. 

11. The apparatus according to claim 10, wherein the 
number of coding system descriptor codes trans- 
mitted immediately after communication is started 
is set in accordance with a memory capacity of the 
partner terminal and, when a user wishes to use a 
coding system other than the previously transmitted 
coding systems, one of the already transmitted cod- 
ing systems is made invalid and the coding system 
descriptor code to be used instead is transmitted 
and made valid. 

12. The apparatus according to claim 1 0, wherein a plu- 
rality of coding system descriptor codes received 
from the partner terminal when communication is 
started or while communication is being performed 
are correctly recognized, and a coding system by 
which received picture coded data is encoded is 
checked for each frame, thereby dynamically 
switching picture coded data decoding means dur- 
ing communication. 

1 3. The apparatus according to claim 1 1 , wherein a plu- 
rality of coding system descriptor codes received 
from the partner terminal when communication is 
started or while communication is being performed 
are correctly recognized, and a coding system by 
which received picture coded data is encoded is 
checked for each frame, thereby dynamically 
switching picture coded data decoding means dur- 
ing communication. 

14. The apparatus according to claim 10, wherein said 
coding system selecting means comprises picture 
encoding means whose number equals the number 
of coding systems for performing picture encoding, 
picture decoding means whose number equals the 
number of coding systems, squared error calculat- 
ing means whose number equals the number of 
coding systems, each of said squared error calcu- 
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lating means calculating a squared error between a 
picture encoded/decoded result and an input pic- 
ture as an original picture, and squared error result 
comparing means for checking which coding sys- 
tem has a smallest squared error, and, in order to s 
select a coding system tor performing coding, cal- 
culates a squared error in each coding system for 
each frame and selects a coding system with a 
smallest squared error. 

10 

15. The apparatus according to claim 11, wherein said 
coding system selecting means comprises picture 
encoding means whose number equals the number 
of coding systems for performing picture coding, 
picture decoding means whose number equals the is 
number ol coding systems, squared error calculat- 
ing means whose number equals the number of 
coding systems, each of said squared error calcu- 
lating means calculating a squared error between a 
picture encoded/decoded result and an input pic- 20 
tureas an original picture, and squared error result 
comparing means for checking which coding sys- 
tem has a smallest squared error, and, in order to 
select a coding system for performing coding, cal- 
culates a squared error in each coding system for 25 
each frame and selects a coding system with a 
smallest squared error. 

16. The apparatus according to claim 12, wherein said 
coding system selecting means comprises picture 30 
encoding means whose number equals the number 

of coding systems for performing picture coding, 
picture decoding means whose number equals the 
number of coding systems, squared error calculat- 
ing means whose number equals the number of 35 
coding systems, each of said squared error calcu- 
lating means calculating a squared error between a 
picture encoded/decoded result and an input pic- 
ture as an original picture, and squared error result 
comparing means for checking which coding sys- 40 
tern has a smallest squared error, and, in order to 
select a coding system for performing coding, cal- 
culates a squared error in each coding system for 
each Irame and selects a coding system with a 
smallest squared error. 



tern has a smallest squared error, and, in order to 
select a coding system for performing coding, cal- 
culates a squared error in each coding system for 
each frame and selects a coding system with a 
smallest squared error. 

18. The apparatus according to any one of claims 1 0 to 
17, wherein said coding system selecting means 
comprises picture adaptive coding system selecting 
means for selecting an appropriate coding system 
in accordance with an input picture, and determines 
a coding system for performing-coding-by checking, 
for each frame, which coding system is appropriate 
to encode a picture. 

1 9. A picture reproducing apparatus in which coded pic- 
ture data is transmitted along with an algorithm for 
adapting a decoding processor according to the 
form in which the data is encoded, said algorithm 
defining a plurality of operation tools for executing 
respective stages of a decoding process, and in 
which information defining a test routine is also 
transmitted, the test routine involving successively 
testing the respective operation tools and in the 
event of a failure ofasaid operation tool proceeding 
to request retransmission of the failed operation tool 
and immediately executing the following operation 
tool test without first re-executing the test on said 
failed operation tool. 



17. The apparatus according to claim 13, wherein said 
coding system selecting means comprises picture 
encoding means whose number equals the number 
of coding systems for performing picture coding, so 
picture decoding means whose number equals the 
number ol coding systems, squared error calculat- 
ing means whose number equals the number of 
coding systems, each of said squared error calcu- 
lating means calculating a squared error between a 55 
picture encoded/decoded result and an input pic- 
ture as an original picture, and squared error result 
comparing means for checking which coding sys- 
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(54) Picture reproducing apparatus 

(57) A picture reproducing apparatus transmits cod- 
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coding) the coded picture data and reconfigures a de- 
coding processor in accordance with the received algo- 
rithm. The apparatus includes a unit for receiving the 
picture reproducing algorithm described in a logical de- 
scription language, performing picture reproduction in 



accordance with the received algorithm, and reconfig- 
uring a picture decoder in accordance with the received 
algorithm, a unit for adding a test routine to the algorithm 
in order to perform an operation test of the received al- 
gorithm in a program download manner, and a picture 
encoder for selecting an optimum coding system for 
each input frame and transmitting a coding system de- 
scriptor code. 
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